Video image coding data transmitter, video image coding data transmission method, video image coding data receiver, and video image coding data transmission and reception system

ABSTRACT

A transmission rate acquisition unit acquires the transmission rate of a network. A predictive transmission structure setting unit sets a transmission structure including a basic hierarchy and a supplementary hierarchy. A first transmission unit transmits basic video image coding data of the basic hierarchy. A memory unit stores supplementary video image coding data of the supplementary hierarchy. A second transmission unit transmits the supplementary video image coding data stored in the memory unit. A transmission control unit controls the second transmission unit according to the transmission rate.

INCORPORATION BY REFERENCE

The disclosure of Japanese Patent Application No. 2014-071975 filed onMar. 31, 2014 including the specification, drawings and abstract isincorporated herein by reference in its entirety.

BACKGROUND

1. Field of the Invention

The present invention relates to a video image coding data transmissiontechnology and, more particularly, to a video image coding datatransmission technology for transmitting video image coding data in realtime.

2. Description of the Related Art

In recent years, services for compression-coding content such as videoimages, sounds, and the like and transmitting compression-coded contentin real time through a network have been growing with the improvement ofthe performance of mobile terminals such as smartphones and the like,the extension of a network band due to the introduction of LTE and thelike, and the progress of a video image compression coding technologysuch as HEVC and the like.

In a network such as the Internet (IP network), best-effort typetransmission is performed, and a band is not guaranteed. Therefore, atransmission rate changes depending on the number of users, the locationof use, and the like. Further, in a wireless network, the network can betemporarily disconnected.

Therefore, TCP, which provides quality assurance such as retransmissioncontrol, is used in non-real time data transmission. However, qualityassurance involving retransmission control is accompanied with delay andis thus not suitable for real-time data transmission. UDP, which doesnot provide quality assurance, is used in real-time data transmission.Conventional technologies for transmitting video images in real time aredisclosed in, for example, Patent document No. 1 and Patent document No.2.

[Patent document No. 1] Japanese Patent Application Publication No.2005-303925

[Patent document No. 2] Japanese Patent Application Publication No.2008-199677

However, a method described in Patent document No. 1 represents atechnology where data selected by thinning selection is transmitted inreal time and unselected data that has not been transmitted is thentransmitted. Thus, a problem with this is that real-time video imagetransmission and viewing of high-quality video images cannot be achievedat the same time. Also, in a method described in Patent document No. 2,a single video image is duplicated for highly-compressed video imagedata and low-compressed video image data and transmitted. Thus, adecrease in transmission efficiency becomes a problem.

SUMMARY

In this background, a purpose of the present invention is to provide avideo image transmission technology capable of achieving real-time videoimage transmission and viewing of high-quality video images at the sametime.

A video image coding data transmitter according to one embodiment of thepresent invention includes: a transmission rate acquisition unit thatacquires the transmission rate of a network; a transmission structuresetting unit that sets a transmission structure including a basichierarchy and a supplementary hierarchy; a first transmission unit thattransmits basic video image coding data of the basic hierarchy; a memoryunit that stores supplementary video image coding data of thesupplementary hierarchy; a second transmission unit that transmits thesupplementary video image coding data stored in the memory unit; and atransmission control unit that controls the second transmission unitaccording to the transmission rate.

Another embodiment of the present invention relates to a video imagecoding data receiver. The video image coding data receiver includes: afirst receiving unit that receives basic video image coding data; avideo image decoding unit that decodes the received basic video imagecoding data so as to reproduce a video image; a second receiving unitthat receives supplementary video image coding data including asupplementary hierarchical picture whose coding order and display orderare earlier than those of a basic hierarchical picture included in thebasic video image coding data; a basic video image coding dataacquisition unit that acquires basic video image coding data receivedbefore supplementary video image coding data that has been received atthe moment; and a video image coding data reconstruction unit thatreconstructs video image coding data from the basic video image codingdata acquired by the basic video image coding data acquisition unit andthe supplementary video image coding data received by the secondreceiving unit.

Yet another embodiment of the present invention relates to a video imagecoding data transmission method. This method includes: acquiring thetransmission rate of a network; setting a transmission structureincluding a basic hierarchy and a supplementary hierarchy; transmittingbasic video image coding data of the basic hierarchy; storingsupplementary video image coding data of the supplementary hierarchy;transmitting the stored supplementary video image coding data; andcontrolling the transmission according to the transmission rate.

Optional combinations of the aforementioned constituting elements andimplementations of the invention in the form of methods, apparatuses,systems, recording mediums, and computer programs may also be practicedas additional modes of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described, by way of example only, withreference to the accompanying drawings that are meant to be exemplary,not limiting, and wherein like elements are numbered alike in severalfigures, in which:

FIG. 1 is a diagram explaining a video image coding data transmissionand reception system according to a first embodiment;

FIG. 2 is a flowchart explaining a procedure of coding and transmittinga video image by a monitoring camera shown in FIG. 1;

FIG. 3 is a diagram explaining a procedure of receiving and decodingvideo image coding data by a video image monitoring station shown inFIG. 1;

FIG. 4 is a diagram explaining an example of a relationship among thesetting of a prediction structure and a transmission structure, basicvideo image coding data, and supplementary video image coding data;

FIG. 5 is a diagram explaining an example of a transmission instructionin the example shown in FIG. 4;

FIG. 6 is a diagram explaining an example of a relationship among thesetting of a prediction structure and a transmission structure, basicvideo image coding data, and supplementary video image coding dataaccording to a second embodiment; and

FIG. 7 is a diagram explaining a video image coding data transmissionand reception system according to a fourth embodiment.

DETAILED DESCRIPTION

The invention will now be described by reference to the preferredembodiments. This does not intend to limit the scope of the presentinvention, but to exemplify the invention.

An explanation will be given hereinafter, along with figures, regardingthe details of a video image coding data transmission technologyaccording to a preferred embodiment of the present invention. In theexplanations of the figures, the same elements shall be denoted by thesame reference numerals, and duplicative explanations will be omitted.

First Embodiment

FIG. 1 is a diagram explaining a video image coding data transmissionand reception system according to a first embodiment. In FIG. 1, threemonitoring cameras 1000 are connected to a video image monitoringstation 2000 via a network (as an example, the Internet). The monitoringcameras 1000 are each an example of a “video image coding datatransmitter,” and the video image monitoring station 2000 is an exampleof a “video image coding data receiver.” A video image captured by themonitoring cameras 1000 is monitored by the video image monitoringstation 2000 in real time.

(Configuration of Monitoring Camera 1000 and Configuration of VideoImage Monitoring Station 2000)

An explanation will be first given regarding the configuration of amonitoring camera 1000. A monitoring camera 1000 is provided with apredictive transmission structure setting unit 100, a video image codingunit 101, a first transmission unit 102, a memory unit 103, a secondtransmission unit 104, a transmission rate acquisition unit 105, and atransmission control unit 106.

The transmission rate acquisition unit 105 acquires the transmissionrate of a network and provides the acquired transmission rate to thetransmission control unit 106 and the predictive transmission structuresetting unit 100. The transmission rate acquisition unit 105 may receiveinformation on the transmission rate via the network from a transmissionstatus information generation unit 205 of the video image monitoringstation 2000 described later or may estimate the transmission rate ofthe network.

The predictive transmission structure setting unit 100 sets thetransmission structure of a video image based on the transmission rateprovided from the transmission rate acquisition unit 105 and also sets aprediction structure that indicates a reference relationship of eachpicture of the video image. The transmission structure of the videoimage includes a basic hierarchy and a supplementary hierarchy. A basichierarchy is a hierarchy for transmitting an overview video image, and asupplementary hierarchy is a hierarchy for transmitting an extensionvideo image.

In this embodiment, the predictive transmission structure setting unit100 sets the ratio of the basic hierarchy and the supplementaryhierarchy according to the transmission rate. Alternatively, thepredictive transmission structure setting unit 100 may set the ratio ofthe basic hierarchy and the supplementary hierarchy to be a defaultvalue or a designated value regardless of the transmission rate.

Based on the transmission structure and the prediction structure set bythe predictive transmission structure setting unit 100, the video imagecoding unit 101 codes a video image that is input from a terminal 1 andgenerates basic video image coding data that corresponds to the basichierarchy and supplementary video image coding data that corresponds tothe supplementary hierarchy. The video image coding unit 101 providesthe basic video image coding data to the first transmission unit 102 andstores the supplementary video image coding data in the memory unit 103.

The first transmission unit 102 transmits the basic video image codingdata provided from the video image coding unit 101 to the video imagemonitoring station 2000 via the network.

Based on the transmission rate provided from the transmission rateacquisition unit 105, the transmission control unit 106 controls thetransmission of the supplementary video image coding data stored in thememory unit 103. When the transmission control unit 106 issues aninstruction for transmitting the supplementary video image coding databased on the transmission rate, the second transmission unit 104 readsout the supplementary video image coding data stored in the memory unit103 and transmits the supplementary video image coding data to the videoimage monitoring station 2000 via the network.

Note that a supplementary hierarchical picture included in thesupplementary video image coding data transmitted by the secondtransmission unit 104 has a coding order and a display order thatprecede those of a basic hierarchical picture included in the basicvideo image coding data transmitted in real time by the firsttransmission unit 102. In other words, the second transmission unit 104belatedly transmits supplementary video image coding data of the pastwhen there is a margin in the transmission rate.

An explanation will be now given regarding the configuration of thevideo image monitoring station 2000. The video image monitoring station2000 is provided with a receiving unit 200, a first video image decodingunit 201, a coding data reconstruction unit 202, a coding data storageunit 203, a second video image decoding unit 204, and a transmissionstatus information generation unit 205.

The receiving unit 200 receives the basic video image coding datatransmitted from the first transmission unit 102 and receives thesupplementary video image coding data transmitted from the secondtransmission unit 104. Note that, while the basic video image codingdata is basic video image coding data coded and transmitted in real timeby the video image coding unit 101 of the monitoring camera 1000, thesupplementary video image coding data is belatedly-transmitted videoimage coding data of the past stored in the memory unit 103 of themonitoring camera 1000. In other words, the supplementary hierarchicalpicture included in the supplementary video image coding datatransmitted from the second transmission unit 104 has a coding order anda display order that precede those of the basic hierarchical pictureincluded in the basic video image coding data transmitted in real timefrom the first transmission unit 102.

The transmission status information generation unit 205 obtains thetransmission status of the network based on the video image coding datareceived by the receiving unit 200, generates transmission statusinformation such as a transmission rate, an error incidence rate, andthe like, and transmits the transmission status information to themonitoring camera 1000 via the network.

The receiving unit 200 provides the received basic video image codingdata to the first video image decoding unit 201 and stores the receivedbasic video image coding data in a memory of the coding datareconstruction unit 202. The receiving unit 200 provides the receivedsupplementary video image coding data to the coding data reconstructionunit 202.

The first video image decoding unit 201 decodes the basic video imagecoding data provided from the receiving unit 200 and outputs the decodedbasic video image coding data to a terminal 2. An overview video imagewith a low frame rate is output in real time from the terminal 2.

The coding data reconstruction unit 202 rearranges the order of picturesbetween the basic video image coding data of the past stored in thememory and the supplementary video image coding data received by thereceiving unit 200 so as to reconstruct video image coding data andstores the reconstructed video image coding data in the coding datastorage unit 203. Note that, since the supplementary hierarchicalpicture included in the supplementary video image coding datatransmitted from the second transmission unit 104 has a coding order anda display order that precede those of the basic hierarchical pictureincluded in the basic video image coding data transmitted in real timefrom the first transmission unit 102 as described previously, the videoimage coding data cannot be reconstructed in combination with thesupplementary video image coding data if the basic video image codingdata of the past is not stored in the memory. The coding datareconstruction unit 202 deletes the basic video image coding data of thepast from the memory after the video image coding data is reconstructed.

The second video image decoding unit 204 reads out and decodes thereconstructed video image coding data from the coding data storage unit203 and outputs the decoded video image coding data to a terminal 3. Anextension video image with a high frame rate is output in non-real timefrom the terminal 3.

The monitoring camera 1000 and the video image monitoring station 2000are realized by the cooperation of hardware and software in aninformation processing device or the like provided with a CPU (CentralProcessing Unit), a frame memory, a hard disk, and the like. By theoperation of the above constituting elements, the monitoring camera 1000and the video image monitoring station 2000 achieve functionalconstituting elements explained in the following.

(Overview of Operation of Monitoring Camera 1000)

An explanation will now be given regarding the overview of the operationof a monitoring camera 1000. FIG. 2 is a flowchart explaining aprocedure of coding and transmitting a video image by a monitoringcamera 1000. An explanation will be given in the following regarding theoperation of the monitoring camera 1000 based on FIG. 2.

The transmission rate acquisition unit 105 acquires the transmissionrate of the network from the transmission status information generationunit 205 via the network (S1000). The acquired transmission rate isprovided to the predictive transmission structure setting unit 100 andthe transmission control unit 106. In this case, the transmission rateacquisition unit 105 acquires the transmission rate of the network fromthe transmission status information generation unit 205 via the network.Alternatively, the transmission rate acquisition unit 105 may monitorthe network and estimate the transmission rate.

The predictive transmission structure setting unit 100 sets a predictionstructure and a transmission structure based on the transmission rateprovided by the transmission rate acquisition unit 105 (S1001). Theprediction structure and the transmission structure that are set areprovided to the video image coding unit 101. The setting of theprediction structure and the transmission structure is performed for apredetermined number of pictures. An explanation will be given on thecondition that the predetermined number is 10. The predetermined numberis simply required to be an arbitrary positive integer such as 15, 30,60, or the like. Loop processing is performed for the predeterminednumber of times for steps S1002 through S1009. The bit rate of the videoimage coding data may be set depending on the transmission rate providedby the transmission rate acquisition unit 105.

The video image coding unit 101 codes a video image that is input fromthe terminal 1 into video image coding data based on the predictionstructure and the transmission structure that are set by the predictivetransmission structure setting unit 100 (S1003). In this case, the videoimage coding unit 101 codes a video image based on HEVC, which is theinternational standard. Alternatively, the video image coding unit 101may code a video image based on MPEG-4AVC/H. 264 (hereinafter, AVC) orthe like. The video image coding data is sorted into basic video imagecoding data and supplementary video image coding data based on thetransmission structure. A description will be made later regarding arelationship among the setting of the prediction structure and thetransmission structure, the basic video image coding data, and thesupplementary video image coding data.

The video image coding unit 101 checks whether the video image codingdata is basic video image coding data (S1004). If the transmissionstructure of the video image indicates that the video image coding datais basic video image coding data (Y in S1004), the video image codingdata is provided to the first transmission unit 102 as basic video imagecoding data, and the first transmission unit 102 transmits the basicvideo image coding data provided by the video image coding unit 101 tothe network in real time (S1005). At this time, UDP is used as atransport layer of the Internet, and a port number PA is assigned as aport.

If the transmission structure of the video image indicates that thevideo image coding data is supplementary video image coding data (N inS1004), the video image coding data is provided to the memory unit 103as supplementary video image coding data, and the memory unit 103 storesthe supplementary video image coding data provided by the video imagecoding unit 101 (S1006).

The memory unit 103 checks whether a transmission instruction providedfrom the transmission control unit 106 is ON (S1007). If thetransmission instruction is ON (Y in S1007), the memory unit 103provides supplementary video image coding data to the secondtransmission unit 104 in the order stored if there is supplementaryvideo image coding data that is stored, and the second transmission unit104 transmits the supplementary video image coding data provided by thememory unit 103 to the network (S1008). Supplementary video image codingdata that is transmitted from the network is deleted from the memoryunit 103. At this time, UDP is used as the transport layer of theInternet, and a port number PB is assigned as a port. The transmissioncontrol unit 106 provides a transmission instruction when there is amargin in a transmission rate provided by the transmission rateacquisition unit 105. A description will be made later regardingconditions and the details of the transmission instruction when there isa margin in the transmission rate. If the transmission instruction isnot ON (N in S1007), the step proceeds to the step S1009 without thetransmission of the supplementary video image coding data.

In order to match the transmission of the supplementary video imagecoding data to the network with the timing of the setting of thetransmission rate, the steps S1007 and S1008 may be performed after thestep S1009.

The operation explained above is repeatedly performed during apredetermined transmission rate period. Therefore, changes in thetransmission rate can be addressed for each predetermined transmissionrate period.

(Overview of Operation of Video Image Monitoring Station 2000)

An explanation will now be given regarding the overview of the operationof a video image monitoring station 2000. FIG. 3 is a diagram explaininga procedure of receiving and decoding video image coding data by a videoimage monitoring station 2000. An explanation will be given in thefollowing regarding the operation of the video image monitoring station2000 based on FIG. 3.

If video image coding data the receiving unit 200 receives from thenetwork is basic video image coding data that is received at a port PA(Y in S1100), the receiving unit 200 provides the basic video imagecoding data to the first video image decoding unit 201 and the codingdata reconstruction unit 202. The first video image decoding unit 201decodes the basic video image coding data provided from the receivingunit 200 so as to generate a reproduction video image and outputs thereproduction video image from the terminal 2 (S1101). Meanwhile, thecoding data reconstruction unit 202 stores the basic video image codingdata provided from the receiving unit 200 in a memory inside the codingdata reconstruction unit 202 (S1102). Alternatively, the coding datareconstruction unit 202 may store the basic video image coding data inan external memory. In short, the coding data reconstruction unit 202 issimply required to be able to acquire basic video image coding datareceived before supplementary video image coding data that has beenreceived at the moment. The step then proceeds to step S1106. Details ofthe step S1106 will be described later.

If the video image coding data the receiving unit 200 receives from thenetwork is not basic video image coding data that is input at the portPA (N in S1100) and there is supplementary video image coding data thatis received at a port PB (Y in S1103), the receiving unit 200 providesthe supplementary video image coding data to the coding datareconstruction unit 202. The coding data reconstruction unit 202reconstructs video image coding data by rearranging the basic videoimage coding data stored in the memory in the step S1102 and thesupplementary video image coding data received in the step S1103(S1104). The reconstructed video image coding data is provided to thecoding data storage unit 203. Subsequently, the coding datareconstruction unit 202 deletes the basic video image coding data storedin the memory from the memory inside the coding data reconstruction unit202 (S1105). The step then proceeds to the step S1106. Details of thestep S1106 will be described later.

If the video image coding data the receiving unit 200 receives from thenetwork is not the basic video image coding data that is input at theport PA (N in S1100) and there is no supplementary video image codingdata that is received at a port PB (N in S1103), the step proceeds tothe step S1106.

In this case, the rearrangement of the basic video image coding data andthe supplementary video image coding data is performed based on a framenumber showing the coding order of pictures that is defined in AVC.Alternatively, a sequence number may be coded in an area different fromthat of the video image coding data such as a UDP extended area. Also,the reconstructed video image coding data may be packetized in afixed-length size just like an MPEG-2 transport stream, multiplexed withsound data or the like, and stored as a media stream. Alternatively, thereconstructed video image coding data may be packetized in units ofaccess units just like an MP4 file, multiplexed with sound data or thelike, and stored as a media stream. Further, while storing the basicvideo image coding data and the supplementary video image coding data inthe order received without rearranging the basic video image coding dataand the supplementary video image coding data, a picture reproductionorder list based on the frame number showing the coding order ofpictures can also be generated and stored. In this case, reproductionbased on the picture reproduction order list allows the video imagecoding data to be properly reproduced without any rearrangement.

The transmission status information generation unit 205 analyzes thestatus of receiving the basic video image coding data and thesupplementary video image coding data of the receiving unit 200 andestimates the transmission rate of the network (S1106). It is assumedthat the method of estimating the transmission rate is a general methodwhere estimation is performed by measuring round trip time. Theestimated transmission rate is provided to the transmission rateacquisition unit 105 of the monitoring camera 1000. The estimatedtransmission rate is assumed to be coded in a UDP extended area.

A bitstream stored in the coding data storage unit 203 is provided tothe second video image decoding unit 204 as necessary and decoded by thesecond video image decoding unit 204 so as to generate a reproductionvideo image, and the reproduction video image is output from theterminal 3.

As described, a reproduction video image with a high frame rate isoutput in non-real time from the terminal 3 while a reproduction videoimage with a low frame rate is output in real time from the terminal 2.

(Event Basis Operation of Video Image Monitoring Station 2000)

An explanation will be now given regarding an event basis operation ofthe video image monitoring station 2000. An event basis operation isperformed by the operation by a user. For example, the event basisoperation is used when the user wishes to see a video image with ahigher frame rate. In this case, the second video image decoding unit204 decodes the video image coding data provided from the coding datareconstruction unit 202 so as to generate a reproduction video image andoutputs the reproduced video image from the terminal 3. The video imagecoding data output from the coding data storage unit 203 may be storedin a file or the like at this time.

(Setting of Prediction Structure and Transmission Structure, Basic VideoImage Coding Data, and Supplementary Video Image Coding Data)

Next, in reference to FIG. 4, an explanation will be given regarding anexample of a relationship among the setting of a prediction structureand a transmission structure, basic video image coding data, andsupplementary video image coding data.

A transmission rate period is a period when a transmission rate acquiredby the transmission rate acquisition unit 105 is effective. It isassumed that the transmission rate that has once acquired does notchange during the transmission rate period. One transmission rate periodis set to include ten pictures. There are four transmission rateperiods: T1, T2, T3, and T4, and respective transmission rates of thetransmission rates periods are R1, R2, R3, and R4. In an example shownin FIG. 4, R1 is 4 Mbps, R2 is 2 Mbps, R3 is 800 kbps, and R4 is 6 Mbps.

In order to facilitate the explanation, an explanation will be givenherein on the assumption that a transmission rate period is the same asa period during which a prediction structure and a transmissionstructure are set (in which the number of pictures for which the sameprediction and transmission structures are applied). However, since atransmission rate period and a period for setting prediction andtransmission structures are independent concepts, these periods may beset separately.

A picture number is assigned to each picture that constitutes a videoimage that is input from the terminal 1. Picture types include an Ipicture for performing intra coding, a P picture for performing interprediction coding, and a B picture for performing inter predictioncoding in a bidirectional manner. In order to simplify the explanation,an I picture and a P picture are used herein for the explanation.However, instead of a P picture, a B picture may be used as a picturetype.

There are two types of transmission structures: a basic hierarchicalpicture (A) and a supplementary hierarchical picture (B). A basichierarchical picture is provided to the first transmission unit 102 asbasic video image coding data, and a supplementary hierarchical pictureis provided to the memory unit 103 as supplementary video image codingdata. A reference picture is a picture used for inter prediction, and areference relationship among pictures is referred to as a predictionstructure. The maximum number of reference pictures is set to be two inthis case.

All pictures are coded as basic hierarchical pictures in thetransmission rate period T1 and the transmission rate period T4.Pictures are coded alternately as basic hierarchical pictures and assupplementary hierarchical pictures in the transmission rate period T2.Pictures are coded alternately as basic hierarchical pictures and assupplementary hierarchical pictures in the ratio of 1:2 or 1:3,respectively, in the transmission rate period T3. The ratio of basichierarchical pictures and supplementary hierarchical pictures has beenexplained to be up to 1:3 thus far. However, the ratio of basichierarchical pictures and supplementary hierarchical pictures can be setto be 1:30, 2:1, etc.

As described, the proportion of basic hierarchies to supplementaryhierarchies can be changed in accordance with the transmission rate.When the transmission rate is sufficiently high (there is a margin), allbasic hierarchies in the transmission rate are coded as basichierarchical pictures. As the transmission rate becomes lowered (thereis less margin), supplementary hierarchical pictures are increased, andthe ratio of basic hierarchical pictures is reduced.

A basic hierarchical picture refers to two immediately preceding basichierarchical pictures at this time. Also, a supplementary hierarchicalpicture refers to two immediately preceding basic hierarchical picturesand/or supplementary hierarchical pictures. As described, limiting basichierarchical pictures to refer to only basic hierarchical picturesallows for decoding in the presence of only basic video image codingdata. Meanwhile, allowing supplementary hierarchical pictures to referto both basic hierarchical pictures and supplementary hierarchicalpictures increases the coding efficiency.

As an example for setting a transmission structure according to atransmission rate, all pictures are coded as basic hierarchical pictureswhen the transmission rate is 4 Mbps or higher, basic hierarchicalpictures and supplementary hierarchical pictures are alternately codedwhen the transmission rate is 1 Mbps or higher and less than 4 Mbps, andbasic hierarchical pictures and supplementary hierarchical pictures arecoded at a proportion of 1:2 or 1:3 when the transmission rate is lessthan 1 Mbps. A minimum transmission rate at which all pictures are codedas basic hierarchical pictures is referred to as a referencetransmission rate. In the example, 4 Mbps is the reference transmissionrate. A case where there is a margin in the transmission rate means acase where the transmission rate is larger than the referencetransmission rate.

As described above, by acquiring a transmission rate for a predeterminedtransmission rate period and by setting the proportion of basichierarchical pictures to supplementary hierarchical pictures such thatthe ratio of the basic hierarchical pictures is increased as thetransmission rate becomes larger and the ratio of the supplementaryhierarchical pictures is increased as the transmission rate becomessmaller, the basic hierarchical pictures can be transmitted in real timeeven when the transmission rate is small.

Further, by transmitting supplementary hierarchical pictures along withbasic hierarchical pictures during a transmission rate period in whichthere is a margin in a transmission rate and by reconstructing videoimage coding data from the basic hierarchical pictures and thesupplementary hierarchical pictures on the receiving side, the viewingof a video image at a low frame rate can be achieved in real time beforethe supplementary hierarchical pictures are received, and the viewing ofa video image at a high frame rate can be achieved after thesupplementary hierarchical pictures are received. Since no duplicatepicture is transmitted in the basic hierarchical pictures and thesupplementary hierarchical pictures, the transmission efficiency is notlowered.

In the above, basic hierarchical coding data and supplementaryhierarchical coding data are transmitted by different ports of UDP.However, as long as the basic hierarchical coding data and thesupplementary hierarchical coding data can be distinguished from eachother in the video image monitoring station 2000, the method is notlimited to this. For example, the basic hierarchical coding data and thesupplementary hierarchical coding data may be respectively assigned withdifferent identifiers, multiplexed in a MP4 file or MPEG-2TS, andtransmitted by a single port. In this case, the identifiers may beprovided in the video image coding data or may be provided at the timeof multiplexing.

In the above, a transmission rate period and/or a period for settingprediction and transmission structures are explained as fixed periods.However, these periods may be variable periods. For example, when thetransmission rate is smaller than a predetermined threshold value orwhen the number of supplementary hierarchical pictures stored in thememory unit 103 is larger than the predetermined threshold value, thetransmission rate period and/or the period for setting the predictionand transmission structures may be controlled to be shorter. When thetransmission rate is smaller than the predetermined threshold value andthe number of the supplementary hierarchical pictures stored in thememory unit 103 is larger than the predetermined threshold value, thetransmission rate period and/or the period for setting the predictionand transmission structures may be controlled to be shorter.

When the transmission rate is large enough, the transmission rate periodand/or the period for setting the prediction and transmission structuresmay be set to be long, and coding may be continued for a while on theassumption that the transmission rate will not change for a while.However, when the transmission rate is small, it is desired, consideringthe possibility of a change in the transmission rate, to set thetransmission rate period and/or the period for setting the predictionand transmission structures to be short, to acquire the latesttransmission rate, and to update the transmission structure frequently.When the number of supplementary hierarchical pictures stored in thememory unit 103 is large, buffer overflow is more likely to occur. Thus,it is desired to set the transmission rate period and/or the period forsetting the prediction and transmission structures to be short, toacquire the latest transmission rate, and to change the transmissionstructure. By setting the transmission rate period and/or the period forsetting the prediction and transmission structures to be variableperiods as described above, the transmission of the supplementaryhierarchical pictures can be properly performed while appropriatelyupdating the transmission rate or appropriately changing thetransmission structure.

Also, a simpler structure can be employed where basic hierarchicalpictures and supplementary hierarchical pictures are set to be Ipictures and P pictures, respectively, and an interval between Ipictures can be adjusted in accordance with the transmission rate forthe purpose of improving the quality of a video image. For example, aninterval between I pictures is controlled such that the interval betweenI pictures is set to be shorter as the transmission rate becomes higherand that the interval between I pictures is set to be longer as thetransmission rate becomes lower. In this manner, basic hierarchicalpictures can be transmitted in real time even when a transmission rateis small with a simpler structure, and supplementary hierarchicalpictures can be also transmitted in a transmission rate period wherethere is a margin in a transmission rate.

(Example of Transmission Instruction)

An explanation will now be given of an example of a transmissioninstruction with reference to FIG. 5. An explanation will be givenherein regarding the example of the transmission instruction using, asan example, the relationship among the setting of a prediction structureand a transmission structure, basic video image coding data, andsupplementary video image coding data shown in FIG. 4. The same as inFIG. 4 applies to a picture number, a picture type, and a transmissionstructure. “Port PA transmission” indicates basic hierarchical picturestransmitted from the port PA, and “Port PB transmission” indicatessupplementary hierarchical pictures transmitted from the port PB.Picture number′ indicates respective picture numbers of thesupplementary hierarchical pictures transmitted from the port PB. In thesame way as in FIG. 4, R1 is 4 Mbps, R2 is 2 Mbps, R3 is 800 kbps, andR4 is 6 Mbps for transmission rates.

The basic hierarchical pictures are transmitted in real time from theport PA. When there is a margin in the transmission rate (in a casewhere the transmission rate is R4), the supplementary hierarchicalpictures are transmitted from the port PB in the order stored withoutexceeding the transmission rate. FIG. 5 shows a state where pictures 11,13, 15, 17, 19, and 21 out of pictures 11, 13, 15, 17, 19, 21, 22, 24,25, 27, 28, and 29, which are the supplementary hierarchical picturesstored in the memory unit 103, are being transmitted in a transmissionrate period T4. As described, supplementary hierarchical pictures havingcoding order and display order that are earlier than those of basichierarchical pictures transmitted in a given transmission interval aretransmitted.

The coding data reconstruction unit 202 rearranges the order of basichierarchical pictures of the past stored in the memory and thesupplementary hierarchical pictures received in the transmission rateperiod T4 and performs reconstruction.

This allows, after supplementary hierarchical pictures are received,video image coding data to be reconstructed from basic hierarchicalpictures stored in the memory and the received supplementaryhierarchical pictures and a video image consisting of all pictures to beviewed, while viewing basic hierarchical pictures in real time.

For example, in a crime prevention system, it is effective to use avideo image with a high frame rate as an evidence video image includinga decisive moment after an event while monitoring a video image with alow frame rate in real time. In mobile communication where a pluralityof transmission paths are available such as 3G/LTE, WiFi, and the like,by reducing a transmission quantity by transmitting only basichierarchical pictures in a 3G/LTE area having a band limitation andtransmitting supplementary hierarchical pictures in a WiFi area havingno band limitation, an overview video image can be reproduced even in anenvironment with a band limitation, and a video image with a high framerate can be reproduced in an environment with no band limitation.

[Exemplary Variation of First Embodiment]

An explanation will be given in the following regarding differences inan exemplary variation of the first embodiment from the firstembodiment. The exemplary variation is different from the firstembodiment in the setting of a transmission structure. In the firstembodiment, basic video image coding data and supplementary video imagecoding data are set in units of pictures. In the exemplary variation,basic video image coding data (basic hierarchical GOP) and supplementaryvideo image coding data (supplementary hierarchical GOP) are set inunits of GOPs.

Regarding the basic hierarchical GOP, the basic hierarchical GOP is setto be an open GOP if the immediately preceding GOP is a basichierarchical GOP, and the basic hierarchical GOP is set to be a closedGOP if the immediately preceding GOP is a supplementary hierarchicalGOP. In a closed GOP, coding and decoding can be performed withreference to pictures in the GOP. On the other hand, a supplementaryhierarchical GOP is always set to be an open GOP. As described, settinga basic hierarchical GOP to be an open GOP only when the immediatelypreceding GOP is a basic hierarchical GOP allows for decoding in thepresence of only a basic hierarchical GOP. Meanwhile, allowing asupplementary hierarchical GOP to be always set to be an open GOPincreases the coding efficiency.

By employing a GOP, which is a unit that is larger than a picture, forthe setting of a transmission structure, a clock that synthesizes basicvideo image coding data and supplementary video image coding data of thecoding data reconstruction unit 202 in the video image monitoringstation 2000 can be reduced. Obviously, a transmission structure can bealso set in units of a plurality of GOPs.

Second embodiment

An explanation will be given in the following regarding a secondembodiment. An explanation will be given mainly of differences from thefirst embodiment. FIG. 6 is a diagram explaining an example of arelationship among the setting of a prediction structure and atransmission structure, basic video image coding data, and supplementaryvideo image coding data according to a second embodiment. As shown inFIG. 6, the second embodiment is different from the first embodiment inthat a transmission structure is divided in a picture in the secondembodiment.

There are two types of transmission structures: a basic hierarchy area(A) and a supplementary hierarchy area (B). A basic hierarchy area iscoded into basic video image coding data and provided to the firsttransmission unit 102, and a supplementary hierarchy area is coded intosupplementary video image coding data and provided to the memory unit103.

An explanation will be first given regarding the monitoring camera 1000.The predictive transmission structure setting unit 100 sets a predictionstructure and a transmission structure based on the transmission rateprovided by the transmission rate acquisition unit 105 (S1001). Anexplanation will now be given regarding the setting of a transmissionstructure according to the second embodiment. A picture is divided inquarters in each of horizontal and vertical directions, forming 16divided areas. A basic hierarchy area and a supplementary hierarchy areaare set for each of the divided areas.

All divided areas are set to be basic hierarchy areas in a transmissionrate period T1 (transmission rate R1) and a transmission rate period T4(transmission rate R4). In a transmission rate period T2 (transmissionrate R2), divided areas in the second and third lines are set to bebasic hierarchy areas, and divided areas in the first and fourth linesare set to be supplementary hierarchy areas. In a transmission rateperiod T3 (transmission rate R3), four divided areas in the center ofthe picture are set to be basic hierarchy areas, and divided areas otherthan the basic hierarchy areas are set to be supplementary hierarchyareas.

The video image coding unit 101 divides into areas a video image that isinput from the terminal 1 based on a transmission structure that is setand codes divided areas into basic hierarchical coding data orsupplementary hierarchical coding data in units of tiles (may be slices)(S1003).

A basic hierarchy area refers to respective basic hierarchy areas of twoimmediately preceding pictures at this time. A supplementary hierarchyarea refers to respective basic hierarchy areas and supplementaryhierarchy areas of the two immediately preceding pictures. Bydetermining a reference relationship as described, an accurate videoimage can be decoded for a basic hierarchy area from basic hierarchyareas only without depending on supplementary hierarchy areas.

In the above, basic hierarchy areas and supplementary hierarchy areasare set as shown in FIG. 6. However, as long as these areas can be setsuch that the basic hierarchy areas become larger (the number of areasdivided as basic hierarchy areas becomes larger) as a transmission ratebecomes larger and that the basic hierarchy areas become smaller (thenumber of areas divided as basic hierarchy areas becomes smaller) as thetransmission rate becomes smaller, the way of setting the areas is notlimited to this. For example, by enlarging a feature area as thetransmission rate becomes larger in combination with a feature areaextraction process, the feature area can be adaptively transmitted inaccordance with the transmission rate.

An explanation will be now given regarding the video image monitoringstation 2000. In the video image monitoring station 2000, a method forreconstructing video image coding data is different. In this case, therearrangement of the basic video image coding data and the supplementaryvideo image coding data is performed based on a macroblock address,which is information showing the position of the first macroblock of aslice. Alternatively, an area position may be coded in an area differentfrom that of the video image coding data such as a UDP extended area,and the rearrangement may be performed based on the area position.

As described above, by transmitting a basic hierarchy area as basichierarchical coding data and transmitting a supplementary hierarchy areaas supplementary hierarchical coding data, the supplementary hierarchyarea can be transmitted in a transmission rate period where there is amargin in a transmission rate while transmitting the basic hierarchyarea in real time. By this, after supplementary hierarchical coding datais received, video image coding data can be reconstructed from basichierarchical coding data and the supplementary hierarchical coding data,and a video image consisting of all areas can be viewed while viewing abasic hierarchy area in real time.

Third embodiment

An explanation will be given in the following regarding a thirdembodiment. An explanation will be given mainly of differences from thesecond embodiment. There are two types of transmission structures in thethird embodiment: basic hierarchical data (A) and supplementaryhierarchical data (B). Basic hierarchical data is provided to the firsttransmission unit 102 as basic video image coding data, andsupplementary hierarchical data is provided to the memory unit 103 assupplementary video image coding data.

The predictive transmission structure setting unit 100 sets a predictionstructure and a transmission structure based on the transmission rateprovided by the transmission rate acquisition unit 105 (S1001). Anexplanation will now be given regarding the setting of a transmissionstructure according to the third embodiment. Video image coding data isclassified syntax-wise into a header, prediction information data, andprediction error data. Further, the prediction error data is classifiedinto low frequency prediction error data and high frequency predictionerror data. Low frequency prediction error data represents informationregarding a direct-current component of orthogonal transformation, andhigh frequency prediction error data represents information regarding analternating-current component of orthogonal transformation.

In a transmission rate period T1 (transmission rate R1) and atransmission rate period T4 (transmission rate R4), a header, predictioninformation data, and prediction error data are set to be basichierarchical data. In a transmission rate period T2 (transmission rateR2), a header, prediction information data, and prediction error dataare set to be basic hierarchical data, and high frequency predictionerror data is set to be supplementary hierarchical data. In atransmission rate period T3 (transmission rate R3), a header andprediction information data are set to be basic hierarchical data, andprediction information data is set to be supplementary hierarchicaldata.

The video image coding unit 101 codes a video image that is input fromthe terminal 1 into basic hierarchical coding data and supplementaryhierarchical coding data based on a transmission structure (S1003).

In the above, video image coding data is classified syntax-wise into aheader, prediction information data, and prediction error data, and theheader, the prediction information data, and the prediction error dataare each set to be basic hierarchical data or supplementary hierarchicaldata. However, this is non-limiting. For example, inter prediction andorthogonal transformation are performed in a variable size in AVC or thelike. Thus, setting to basic hierarchical data and supplementaryhierarchical data can be performed based on a block size in the same wayas, for example, setting a block having a predetermined block size orless to be basic hierarchical data and a block having the predeterminedblock size or larger to be supplementary hierarchical data (the basichierarchical data and the supplementary hierarchical data may beswitched for the setting). As a block size in this case, a block sizefor inter prediction may be selected when the quality of a video imageto be transmitted is desired to be controlled based on the magnitude ofmotion in a video image, and a block size for orthogonal transformationmay be selected when the quality of a video image to be transmitted isdesired to be controlled based on the priority in the magnitude of aprediction error. This allows the proportion of basic hierarchicalcoding data to supplementary hierarchical coding data to be set flexiblyaccording to the priority in the quality of a video image to betransmitted.

As described above, by transmitting a part of the syntax of video imagecoding data as basic hierarchical coding data and transmitting syntaxfor areas other than a basic hierarchy area as supplementaryhierarchical coding data, the video image coding data can bereconstructed from the basic hierarchical coding data and thesupplementary hierarchical coding data after the supplementaryhierarchical coding data is received, and detailed information of avideo image can be viewed, while viewing the basic information of thevideo image in real time.

[Exemplary Variation of Third Embodiment]

An explanation will be given in the following regarding an exemplaryvariation of the third embodiment. An explanation will be givenregarding differences from the third embodiment. In the exemplaryvariation, the setting of a transmission structure is different fromthat of the third embodiment.

As a first exemplary variation, it is assumed that a luminance componentY represents basic hierarchical coding data and that a first colordifference component Cb and a second color difference component Crrepresent supplementary hierarchical coding data. As described above, bytransmitting a luminance component as basic hierarchical coding data andtransmitting a first color difference component and a second colordifference component as supplementary hierarchical coding data, whileviewing a monochrome video image in real time, video image coding datacan be reconstructed from the basic hierarchical coding data and thesupplementary hierarchical coding data after the supplementaryhierarchical coding data is received so as to view a color video image.

Fourth embodiment

An explanation will be given for a forth embodiment in the followingmainly regarding differences from the first embodiment. FIG. 7 is adiagram explaining a video image coding data transmission and receptionsystem according to the fourth embodiment. In FIG. 7, three monitoringcameras 1100, which represent an example of a video image coding datatransmitter, are connected to a video image coding data relay device1200 via a second network. Further, the video image coding data relaydevice 1200 is connected to the video image monitoring station 2000,which is an example of a video image coding data receiver, via a firstnetwork. An example of the second network is wireless LAN such as WiFi,and an example of the first network is LTE. The video image coding datarelay device 1200 is installed, for example, inside a smartphone as anexample of an LTE relay station.

(Configuration of Monitoring Camera 1100 and Configuration of VideoImage Coding Data Relay Device 1200)

An explanation will be first given regarding the configuration of amonitoring camera 1000. A monitoring camera 1000 is provided with avideo image coding unit 101 and a third transmission unit 107.

Based on a transmission structure and a prediction structure set by apredictive transmission structure setting unit 122 of a video imagecoding data relay device 1200, the video image coding unit 101 codes avideo image that is input from a terminal 1 and generates basic videoimage coding data that corresponds to a basic hierarchy andsupplementary video image coding data that corresponds to asupplementary hierarchy. The video image coding unit 101 provides avideo image stream containing the basic video image coding data and thesupplementary video image coding data to the third transmission unit107.

The third transmission unit 107 transmits the video image streamprovided from the video image coding unit 101 to the video image codingdata relay device 1200 via the second network.

An explanation will now be given regarding the configuration of a videoimage coding data relay device 1200. A video image coding data relaydevice 1200 is provided with a second receiving unit 120, a video imagecoding data analysis unit 121, a predictive transmission structuresetting unit 122, a first transmission unit 102, a memory unit 103, asecond transmission unit 104, a transmission rate acquisition unit 105,and a transmission control unit 106.

The transmission rate acquisition unit 105 acquires the transmissionrate of a first network and provides the acquired transmission rate tothe transmission control unit 106 and the predictive transmissionstructure setting unit 122.

The predictive transmission structure setting unit 122 sets thetransmission structure of a video image based on the transmission rateprovided from the transmission rate acquisition unit 105 and also sets aprediction structure that indicates a reference relationship of eachpicture of the video image.

The second receiving unit 120 receives a video image stream from amonitoring camera 1100 and provides the video image stream to the videoimage coding data analysis unit 121.

Based on the transmission structure and the prediction structure set bythe predictive transmission structure setting unit 122, the video imagecoding data analysis unit 121 separates and extracts basic video imagecoding data and supplementary video image coding data from a video imagestream, provides the basic video image coding data to the firsttransmission unit, and stores the supplementary video image coding datain the memory unit 103. In the above, the basic video image coding dataand the supplementary video image coding data are separated from thevideo image stream based on the transmission structure and theprediction structure. Alternatively, the basic video image coding dataand the supplementary video image coding data can be respectivelyassigned with different identifiers in advance and separated.

The first transmission unit 102 transmits the basic video image codingdata provided from the video image coding data analysis unit 121 to thevideo image monitoring station 2000 via the first network.

Based on the transmission rate provided from the transmission rateacquisition unit 105, the transmission control unit 106 controls thetransmission of the supplementary video image coding data stored in thememory unit 103. When the transmission control unit 106 issues aninstruction for transmitting the supplementary video image coding databased on the transmission rate, the second transmission unit 104 readsout the supplementary video image coding data stored in the memory unit103 and transmits the supplementary video image coding data to the videoimage monitoring station 2000 via the first network.

In the fourth embodiment, although the video image coding data relaydevice 1200 does not perform a process of coding a video image since thevideo image coding data relay device 1200 receives video image codingdata already coded by the monitoring camera 1100, the video image codingdata relay device 1200 performs the same process as in the monitoringcamera 1100 according to the first embodiment for the transmission ofvideo image data after coding. The configuration and operation of thevideo image monitoring station 2000 are the same as those in the firstembodiment.

The video image coding data relay device 1200 is realized by thecooperation of hardware and software in an information processing deviceor the like provided with a CPU (Central Processing Unit), a framememory, a hard disk, and the like. By the operation of the aboveconstituting elements, the video image coding data relay device 1200achieves functional constituting elements explained in the following.

(Overview of Operation of Monitoring Camera 1100)

An explanation will now be given regarding the overview of the operationof the monitoring camera 1100. The prediction structure and thetransmission structure received from the second network are provided tothe video image coding unit 101. The video image coding unit 101generates video image coding data based on the prediction structure andthe transmission structure that are received and provides the videoimage coding data to the third transmission unit 107. The thirdtransmission unit 107 transmits the video image coding data providedfrom the video image coding unit 101 to the second network in real time.

(Overview of Operation of Video Image Coding Data Relay Device 1200)

An explanation will now be given regarding the overview of the operationof the video image coding data relay device 1200.

The predictive transmission structure setting unit 122 sets a predictionstructure and a transmission structure based on the transmission rateprovided by the transmission rate acquisition unit 105. The predictionstructure and the transmission structure that are set are provided tothe video image coding unit 101 via the second network. It is assumedthat the setting of the prediction structure and the transmissionstructure is performed based on, for example, a specification such asOpen Network Video Interface Forum (ONVIF). Further, the predictionstructure and the transmission structure that are set are also providedto the video image coding data analysis unit 121. On the other hand, thesecond receiving unit 120 receives the video image coding datatransmitted from the third transmission unit 107 via the second network.

Based on the prediction structure and the transmission structureprovided by the predictive transmission structure setting unit 122, thevideo image coding data analysis unit 121 separates the video imagecoding data input from the second receiving unit 120 into basic videoimage coding data and supplementary video image coding data, providesthe basic video image coding data to the first transmission unit 102,and provides the supplementary video image coding data to the memoryunit 103. A process of transmitting the basic video image coding dataand the supplementary video image coding data that follows after this isthe same as that in the first embodiment. The basic video image codingdata is transmitted to the video image monitoring station 2000 in realtime via the first network, and the supplementary video image codingdata is transmitted to the video image monitoring station 2000 via thefirst network when there is a margin in the transmission rate of thefirst network.

As described above, in the fourth embodiment, by using a smartphone orthe like that has a transmission and reception function as a video imagecoding data relay device in advance, it is no longer necessary to add afunction of distinguishing a basic hierarchy from a supplementaryhierarchy and transmitting the basic hierarchy from the supplementaryhierarchy to a monitoring camera. Thus, a conventional function of amonitoring camera system can be easily expanded.

The processes described in the above embodiments can obviously beimplemented by hardware-based apparatus for transmission, storage, orreception. Alternatively, the processes can be implemented by firmwarestored in a read-only memory (ROM), a flash memory, etc., or by softwareon a computer, etc. The firmware program or the software program may bemade available on, for example, a computer readable recording medium.Alternatively, the programs may be made available from a server via awired or wireless network. Still alternatively, the programs may be madeavailable in the form of data transmission over terrestrial or satellitedigital broadcast systems.

Described above is an explanation of the present invention based on theembodiments. The embodiments are intended to be illustrative only, andit will be obvious to those skilled in the art that variousmodifications to constituting elements and processes could be developedand that such modifications are also within the scope of the presentinvention.

What is claimed is:
 1. A video image coding data transmitter comprising:a transmission rate acquisition unit that acquires the transmission rateof a network; a transmission structure setting unit that sets atransmission structure including a basic hierarchy and a supplementaryhierarchy; a first transmission unit that transmits basic video imagecoding data of the basic hierarchy; a memory unit that storessupplementary video image coding data of the supplementary hierarchy; asecond transmission unit that transmits the supplementary video imagecoding data stored in the memory unit; and a transmission control unitthat controls the second transmission unit according to the transmissionrate.
 2. The video image coding data transmitter according to claim 1,further comprising: a video image coding data analysis unit thatdistinguishes the basic video image coding data of the basic hierarchyfrom the supplementary video image coding data of the supplementaryhierarchy that are included in video image coding data based on thetransmission structure.
 3. The video image coding data transmitteraccording to claim 1, wherein the transmission structure setting unitsets the proportion of the basic hierarchy to the supplementaryhierarchy such that the ratio of the basic hierarchy is increased as thetransmission rate becomes larger and the ratio of the supplementaryhierarchy is increased as the transmission rate becomes smaller.
 4. Thevideo image coding data transmitter according to claim 1, wherein aperiod for acquiring the transmission rate or a period for setting thetransmission structure is changed according to at least one of thenumber of pictures in the supplementary hierarchy stored in the memoryunit and the transmission rate.
 5. The video image coding datatransmitter according to claim 1, wherein the transmission structuresetting unit sets a part of the area of a picture to represent the basichierarchy and a part of the area other than the part representing thebasic hierarchy to be the supplementary hierarchy.
 6. The video imagecoding data transmitter according to claim 1, wherein the transmissionstructure setting unit sets a part of the syntax of the video imagecoding data to represent the basic hierarchy and a part of the syntaxthat is different from the part representing the basic hierarchy torepresent the supplementary hierarchy.
 7. The video image coding datatransmitter according to claim 1, wherein the transmission structuresetting unit sets a video image consisting of a luminance component torepresent the basic hierarchy and a video image consisting of a colordifference component to represent the supplementary hierarchy.
 8. Avideo image coding data receiver comprising: a first receiving unit thatreceives basic video image coding data; a video image decoding unit thatdecodes the received basic video image coding data so as to reproduce avideo image; a second receiving unit that receives supplementary videoimage coding data including a supplementary hierarchical picture whosecoding order and display order are earlier than those of a basichierarchical picture included in the basic video image coding data; abasic video image coding data acquisition unit that acquires basic videoimage coding data received before supplementary video image coding datathat has been received at the moment; and a video image coding datareconstruction unit that reconstructs video image coding data from thebasic video image coding data acquired by the basic video image codingdata acquisition unit and the supplementary video image coding datareceived by the second receiving unit.
 9. A video image coding datatransmission method comprising: acquiring the transmission rate of anetwork; setting a transmission structure including a basic hierarchyand a supplementary hierarchy; transmitting basic video image codingdata of the basic hierarchy; storing supplementary video image codingdata of the supplementary hierarchy; transmitting the storedsupplementary video image coding data; and controlling the transmissionaccording to the transmission rate.
 10. A video image coding datatransmission program embedded in a non-transitory computer readablerecording medium that, when executed on a computer, implements:acquiring the transmission rate of a network; setting a transmissionstructure including a basic hierarchy and a supplementary hierarchy;coding a video image based on the transmission structure into videoimage coding data including basic video image coding data of the basichierarchy and supplementary video image coding data of the supplementaryhierarchy; transmitting basic video image coding data of the basichierarchy; storing supplementary video image coding data of thesupplementary hierarchy; transmitting the stored supplementary videoimage coding data; and controlling the transmission according to thetransmission rate.
 11. A video image coding data transmission andreception system comprising: a video image coding data transmitter thatcodes and transmits a video image; and a video image coding datareceiver that receives and reconstructs video image coding data, whereinthe video image coding data transmitter includes: a transmission rateacquisition unit that acquires the transmission rate of a network; atransmission structure setting unit that sets a transmission structureincluding a basic hierarchy and a supplementary hierarchy; a firsttransmission unit that transmits basic video image coding data of thebasic hierarchy; a memory unit that stores supplementary video imagecoding data of the supplementary hierarchy; a second transmission unitthat transmits the supplementary video image coding data stored in thememory unit; and a transmission control unit that controls the secondtransmission unit according to the transmission rate, and wherein thevideo image coding data receiver includes: a first receiving unit thatreceives the basic video image coding data; a video image decoding unitthat decodes the received basic video image coding data so as toreproduce a video image; a second receiving unit that receives thesupplementary video image coding data including a supplementaryhierarchical picture having coding order and display order that areearlier than those of a basic hierarchical picture included in the basicvideo image coding data; a basic video image coding data acquisitionunit that acquires basic video image coding data received beforesupplementary video image coding data that is currently received; and avideo image coding data reconstruction unit that reconstructs videoimage coding data from the basic video image coding data acquired by thebasic video image coding data acquisition unit and the supplementaryvideo image coding data received by the second receiving unit.